The future of autonomous driving is slowly approaching, but there are still many steps to take before it can become a reality. It is crucial to pay attention to road infrastructure, because without it, intelligent vehicles will not be able to operate reliably, and it will never be possible to dispense of driver's control. This paper presents the work carried out for the detection of road markings damage using computer vision techniques. This is a complex task for which there are currently not many papers and large image sets in the literature. This study uses images from the public Road Damage Detection dataset for the D44 defect and also provides 971 new labelled images for Spanish roads. For this purpose, three detectors based on deep learning architectures (Faster RCNN, SDD and EfficientDet) have been used and single-source and mixed-source models have been studied to find the model that best fits the target images. Finally, F1-score values reaching 0.929 and 0.934 have been obtained for Japanese and Spanish images respectively which improve the state-of-the-art results by 25%. It can be concluded that the results of this study are promising, although the collection of many more images will be necessary for the scientific community to continue advancing in the future in this field of research.